In general, the present invention is situated in the context of physical object security which forms an important issue since quite some time in several domains, such as pharmaceutical and cosmetics products, electronics, luxury watches, text documents and certificates due to the striking numbers and scale of counterfeiting and its spread worldwide. Despite numerous efforts from brand owners and manufacturers, the end-consumers are not well aware, respectively cannot be well aware of the particularities of the original design of a given product or content which is often changed for various commercial and technical reasons and the entire spectrum of security features applied to protect a particular brand. To protect the consumer, for example against taking a counterfeited drug, and to create an essential element of a global anti-counterfeiting network, it would be highly desirable that users can perform a verification by themselves. Furthermore, it would be desirable that information from such verifications could be stored and analysed, amongst other in order to contribute towards accurate and fast information on geographical trends in appearance and distribution in counterfeiting goods as soon as such items enter the market. A similar situation also exists with respect to banknotes that are protected by quite sophisticated security techniques which, however, may not be verified by the average end-consumer, but only by experts.
The problem is complicated by the fact that over the past several years, counterfeiters have gained access to sophisticated technologies that enable them to closely duplicate the brand products. It is a well known fact that counterfeited products can be so accurately duplicated that it even experts might need some special equipment to verify them. However, counterfeited products in most cases bear, as compared to the genuine products, some differences that can be easily recognized by trained experts but that are at the same time difficult or almost impossible to be recognized by casual consumers.
Previous attempts of the security industry to introduce security technologies based on special detectors had little success. This is due to the fact that it is commercially highly unattractive to produce specific hardware devices, which would need to be distributed and regularly upgraded on a global level, simply for verifying individual products. Moreover, although there is a need for fast authentication of consumer goods, end consumers show little interest in using specialized devices for anti-counterfeiting measures, respectively cannot have specific know-how that might be needed to use such specialized devices.
Existing security mechanisms require to modify the technological approach or manufacturing processes of products, do change the look or properties of products, do add extra features or materials, such as holograms, RFIDs, magnetic, rare-earth or luminescent materials, or do require interventions or invasive procedures. Moreover, such protection mechanisms do not always ensure backward compatibility with products already produced in the past.
In general, there currently exist three main methods allowing to realize product verification and, coming along with that, a kind of consumer protection, which, however, use fundamentally different approaches. One method is digital watermarking, also referred to as data hiding, another method is content fingerprinting, also known as robust hashing, and there finally exist methods based on direct visual inspection.
Digital watermarking or data hiding achieves protection by content modification, in particular by embedding a special mark or encoded message which carries information on the protected content. Some of the methods that implement this functionality to physical documents are described in U.S. Pat. No. 7,991,182 for images and in U.S. Pat. No. 7,644,281 for text documents and vector graphics including barcodes as described in US 2010/0012736. The content should be modified such as to ensure reliable and secure extraction of an embedded, desired message. This has three important consequences concerning the necessity (a) to modify the content or object either prior to manufacturing or reproduction or to introduce the marking to already existing objects, see e.g. U.S. Pat. No. 7,965,862, (b) that it is also clonable, if the object design is quite accurately reproduced, and (c) that all objects should be protected by this technology prior to their distribution which is difficult or almost impossible to realize in practice. In fact, watermarking algorithms are designed to be robust to different imperfections in acquisition, especially on mobile devices. Therefore, even if a fake contains some imperfections, the inherent capability of digital watermarks to correct errors will “mask” such imperfections and reduce the ability to detect the fake. Furthermore, direct product marking is also disclosed in U.S. Pat. No. 7,686,231, where a visible marking is added to the product. However, such product modification often is not acceptable to brands and manufacturers because of modifying the product design and look, or complicating the existing manufacturing pipeline. Similar methods are also known to mark text documents by modifying the character shape and size, elements of characters, printing halftone etc., see e.g. U.S. Pat. No. 7,644,281. Being acceptable for new documents, these methods cannot cope with already produced documents. Therefore, such methods do not provide protection for certain types of documents.
Digital content fingerprinting or robust hashing are techniques which extract a fingerprint directly from the object features and no preliminary content pre-processing is performed. As a result, the fingerprints are not very robust in comparison to digital watermarks. This technique is primarily used for digital media such as video and images, see e.g. US 2009/0292701 and U.S. Pat. No. 7,552,120, as well as audio, see e.g. J. Haitsma, T. Kalker, and J. Oostveen, “Robust audio hashing for con-tent identification,” in Proc. of the Content-Based Multimedia Indexing, Firenze, Italy, September 2001. However, it is also used for physical documents, such as described in in F. Jordan, M. Kutter, C. di Venuto, Means for using microstructure of materials surface as a unique identifier, WO 2007/028799, and R. P. Cowburn, J. D. R. Buchanan, Authenticity verification by means of optical scattering, WO 2006/120398. The lack of robustness leads to a large amount of errors or mismatches between the original fingerprint and the fingerprint extracted from a distorted counterpart of the original object. In turn, this results in the necessity to perform a highly complex search and identification for trying to match the original fingerprint and the extracted fingerprint. As a result it is difficult to distinguish the errors due to the acquisition imperfections from those caused by content modifications. In recent years, methods were developed which use local features designed in the computer vision for object recognition. This is well suited to classification of objects into several predefined categories that rarely exceed 20′000 classes or searching of semantically similar images. However, the local nature of robust features does not make it possible to detect the small or invisible differences in the object design or appearance. These methods are used for content-based retrieval where only rough or semantic similarity is sufficient to find perceptually close objects. Also, the application of fingerprinting methods for characterising materials that are used for the tracking and tracing of individual items is often not suitable due to (a) complexity of acquisition of a fingerprint from each item, (b) the management of huge databases and (c) searching in these databases that can be in the of order of billions of entries. In most cases, even some special imaging devices are needed to capture the structure of materials, see e.g. R. P. Cowburn, J. D. R. Buchanan, Authenticity verification by means of optical scattering, WO 2006/120398.
The use of fingerprinting techniques is also disclosed in US 2012/10324534 for verification of authenticity of an identity document. The fingerprints are extracted from the text parts and photographic images of the digitized documents using specially designed local and global descriptors adapted to facial images. The text parts are recognized by Optical Character Recognition (OCR) in predefined and aligned regions. These are stored in a centralized database. The global descriptors are computed based on image gradients, whereas the local descriptors are based on a binarized source image. These descriptors are suited for the authentication of identity documents, because the facial images on said documents can be well acquired, standardized according to the templates, and do not exhibit a lot of variability in terms of semantic context. However, this approach meets serious technical constraints if applied to generic object identification, where the reproduced information might be highly variable and non-homogeneous. Furthermore, apart from not being adapted for other types of objects than identity documents having a well defined structure, this framework is not adapted to be used on any kind of device, because many devices suffer from non-linear geometric distortions, which impact the stability of the above mentioned descriptors.
Several documents like US 2011/0158483, US 2010/0329576 disclose a system for printed document authentication and alteration detection based on the well-known bag-of-feature principle, see e.g. also D. G. Lowe, Object recognition from local scale invariant features. In the Proc. of 7th International Conference on Computer Vision, Sep. 20-27, 1999. Segmented parts of a document are considered as individual images containing small elements. A codebook is trained on these small elements, which contain text fragments. Documents to be verified that do not match sufficiently against the enrolled elements in the codebook are considered to be fake. Unfortunately, these block-wise features are not invariant to geometrical distortions, thus necessitating some form of pre-alignment. Such an alignment is possible leveraging the intrinsic geometric structure of the lines of text in a document. However, this approach is obviously not applicable for generic objects that lack strict design elements. Additionally, such type of codebook with block-wise patches can be trained on generic fonts and languages and will not exhibit a lot of variability. In contrast, for authenticating any kind of objects graphic design elements will be more distinctive and consequently also require more memory storage, such that the verification procedure will be more computational intensive. On top of that and similar to the previously described method, depending on the acquisition device used to acquire the entire page document, geometric distortions may be introduced in the acquired image patches burdening the system and hurting performance. Finally, document authentication is assumed to be performed on a document whose identity is perfectly known in advance and which might be established based on added bar codes or any marking which is not feasible in many type of applications.
Security techniques based on visual inspection require either (a) perfect knowledge of the object design particularities or (b) in front presence of a design reference template. At large scale, this is difficult to manage, store and search. Moreover, it is impossible to see small imperfections or deviations by the naked eye. Moreover, the need to store and distribute the elements of design with a high level of details in an open form accessible to humans is not well appreciated by brand owners for various security reasons, since leading to an easier clonability of objects as well as a leak of technological know-how to competitors. Known systems for visual off-line quality inspection require high-resolution stationary imaging systems with accurate alignment of samples and light, see for example the disclosure of WO2013/163978.
In short, the above mentioned existing methods comprise several problems for a variety of reasons which comprise the need to introduce modifications, low discriminative capabilities, high complexity of involved procedures, fundamental restrictions of humans to examine variations at microscopic level or simply the lack of desire of doing so for the ordinary consumers, and the fact that product design elements should be securely protected prior to storage in publicly available services or distribution to consumer devices.
The solutions according to prior art therefore do not entirely satisfy nowadays needs with respect to the protection of generic physical—or digital objects, such that there is still a need for a method for automatic, fast, reliable and secure verification of authenticity of objects to the consumer.